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Assuming that a non-standard neutral Higgs with an enhanced Yukawa coupling to a bottom 
quark is observed at future hadron experiments, we propose a method for a better understanding 
of the Higgs sector. Our procedure is based on "counting" the number of events with heavy jets 
(where "heavy" stands for a c or fo jet) versus b jets, in the final state of processes in which the 
Higgs is produced in association with a single high pr cor b jet. We show that an observed signal of 
the type proposed, at either the Tevatron or the LHC, will rule out the popular two Higgs doublet 
model of type II as well as its supersymmeric version - the Minimal Supersymmetric Standard Model 
(MSSM), and may provide new evidence in favor of some more exotic multi Higgs scenarios. As an 
example, we show that in a version of a two Higgs doublet model which naturally accounts for the 
large mass of the top quark, our signal can be easily detected at the LHC within that framework. We 
also find that such a signal may be observable at the upgraded Tevatron Runlll, if the neutral Higgs 
in this model has a mass around 100 GeV and tan /3 xi50 and if the efficiency for distinguishing a c 
jet from a light jet reaches the level of ~ 50%. 
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I. INTRODUCTION 

With the Tevatron Runll underway at Fermilab, the 
future Tevatron upgrade Runlll and the Large Hadron 
Colhder (LHC) at CERN, the discovery of the Higgs 
boson is currently beheved to be "around the corner" . 
These machines will be able to detect the neutral Higgs 
over its entire theoretically allowed mass range 0. 

If the Higgs Yukawa coupling to a bottom quark is en- 
hanced, as predicted for example in general multi Higgs 
models and in the supersymmetric version of the two 
Higgs Doublet Model (2HDM) - the MSSM - when tan/3 
is large, then the neutral Higgs may be produced at Lead- 
ing Order (LO) through bb fusion 66 — > ft, "z]. In such 
models, a sizable production rate of a neutral Higgs in 
association with 6 quark jets is also expected via the sub- 
processes 56 ^ 6/i 0,0 and gg, qq ^ bbh ^.[^1 In par- 
ticular, as was shown in if one demands that only one 
6 jet will be produced (in association with the Higgs) at 
high pt, then gb — > bh becomes the LO and is therefore 
the dominant Higgs-bottom associated production mech- 
anism for large tan/3. For example, gb bh followed by 
h ^ bb may already prove to be a useful probe of the 
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[1] Unless stated otherwise, h will be used to denote the neutral 
Higgs that drives the processes under consideration. 



neutral Higgs sector at the Tevatron future runs 

In this work we assume that indeed the Higgs has an 
enhanced coupling to bottom quarks and that it will be 
therefore discovered or observed in association with bot- 
tom quarks jets, via one of the above production chan- 
nels. Discovery of such a non-standard Higgs should start 
an exploration era of the various Higgs Yukawa couplings 
to fermions in order to decipher the origin and detailed 
properties of the Higgs sector. For that purpose, special 
observables should be devised that will enable the ex- 
perimentalists to pin-down some specific Higgs Yukawa 
interactions. In particular, one should find observables 
which are sensitive to a specific Yukawa coupling or com- 
bination of couplings, and are therefore able to exclude 
models with an enhanced Higgs-bottom Yukawa vertex, 
or provide further evidence in favor of a definite Higgs 
scenario. 

In this paper we follow this line of thought and present 
a method which is useful for an investigation of the rela- 
tion between the Higgs-charm and Higgs-bottom Yukawa 
couplings. We focus on neutral Higgs production via a 
single high px charm or bottom jet, qg — > qh, g = c or 6, 
followed by /i ^ 66 or ft ^ cc.'^l Our final state is there- 
fore defined to be composed out of exactly three non-light 
jets, where a light jet means a. u,d, s ov g (gluon) jet. 
If the Higgs is not the Standard Model (SM) one, par- 



[2] We will not explicitly write the charge of the quarks involved. All 
charge-conjugate sub-processes are understood throughout this 
paper. 
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ticularly if the Higgs-bottom Yukawa is large, then this 
class of processes may play an important role in Higgs 
searches, discovery and characterization at hadron col- 
liders. 

Based on this type of production mechanism, we sug- 
gest an observable which is particularly sensitive to new 
physics in the ratio between the Higgs-charm and the 
Higgs-bottom Yukawa couplings. Wc find that our signal 
is insensitive to the Higgs sectors of the type H 2HDM 
(2HDMH) and of the MSSM. Therefore, a positive sig- 
nal of our observable at either the Tevatron or the LHC 
will rule out these models. We also consider a different 
type of a 2HDM, the so called "2HDM for the top quark" 
(T2HDM), which is an interesting model designed to nat- 
urally accommodate the large mass of the top quark. 
This model may be viewed as an effective low energy 
theory wherein the Yukawa interactions mimic some high 
energy dynamics which generates both the top mass and 
the weak scale. The two scalar doublets could be com- 
posite, as in top-color models, and the Yukawa interac- 
tions could be the residual effect of some higher energy 
four-Fermi operators. Interestingly, for this T2HDM our 
signal is significantly enhanced. We thus show that if 
this version of a 2HDM underlies the Higgs sector, then 
our signal will be easily detected (to many standard de- 
viations) at the LHC and may also be detected at the 
Tevatron if tan (3 is large enough and if the efficiency of 
tagging a c jet as a heavy jet is of 0(50%). 

The paper is organized as follows: in section H we 
define our signal. In section III we present the numeri- 
cal setup and discuss the experimental background. In 
section IV we evaluate our signal in three different ver- 
sions of a 2HDM and discuss the expectations from the 
T2HDM, the 2HDMII and the MSSM. In section V we 
summarize our results. 



II. NOTATION AND DEFINITION OF THE 
SIGNAL 

Let jh denote a "heavy" b or c-quark jet. Specifically, 
a jh jet should not be a u, d, s or gluon jet (i.e., with the 
veto jh u, d, s, g jet). Let jb be a 6-quark jet. Clearly, 
every jet of type jb is also a jh ■ Likewise, let us denote 
by jc a c-quark jet. 

We thus define the cross-sections 



and the ratio 



a{pp or pp jhjhjh + X) , (1) 
(j{pp or pp jbjbjb + X) , (2) 



cr-i 



From the above definitions it is evident that: 
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where and Yb are the Higgs-charm and Higgs-bottom 
Yukawa couplings, respectively. 

In reality, one has to include non-ideal efficiencies for 
jet identification. Therefore, we define the quantity e*^ to 
be the efficiency for identifying a j jet as a fc jet. Thus, 
for example, or ej* are the efficiencies for misidentifying 
a fight jet (j;) as a 5 jet {jb) or as a heavy jet {jh), 
respectively. Using these efficiency factors we define the 
experimental observable as: 



R 



M 



JbJbJb 



where R = a 



JhJhJh/ '^jbjbjb with (Jjhjf^jh: 



(6) 



<^3b3b3b being the 



effective signals for cross-sections that will actually be 
measured in the experiment and R^ is the value of R 
calculated within a specific Higgs model M via the sub- 
processes gq —>■ qh —>■ qqq, q = cor b. Thus, in the limit of 
an ideal detector where — S^, i?/*^ = 7^*^ (where TZ^^ 
means the "ideal" ratio defined in ||3Jl calculated within 
model M). In particular, these cross-sections include all 
tagging efficiencies as follows: 



-M 



b b M 
32^ js 313233 ' 
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In the limit (Yc/Yb)'^ 0, one obtains 
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b3b 



(9) 

Therefore, due to the non-ideal efficiencies (in particular 
if 7^ e^), we have i?" ^ 1 even if the Higgs Yukawa 
coupling to the charm quark is negligible. As will be 
shown below, this limit is applicable to a neutral Higgs 
either of MSSM or of 2HDMII origin, if tan (3 is large. 
That is, for large tan/3 



_ "3h3h3h 

'^jbjbjb 



(3) 



Note that, since jh is either a jb or a jc, it follows that: 



j^MSSM j^HDMII 



(10) 



A deviation from R^^ = indicating a high rate for 
h — > cc, signals a specific type of new physics in the Higgs 
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sector, e.g., beyond the MSSM or the 2HDMII Higgs sce- 
nario. Such a deviation is parameterized by: 



Yb 



(11) 



The significance with which SR can be measured de- 
pends on the experimental error SR^^ : 



^R^p - R 



M 



3h3h3h ) 



(ATM. . )2 



]b]b3b' 
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where Nf^- ■ = af^,,^ x L and similarly for Nf^- - 

3hjh3h 3h3h3h ^ Jbjb3h 

with L being the integrated luminosity at the given col- 
lider. The errors in the measurements of iV*^,- and 

^3^b3b3b ai^e (st atistical only): AA^j,j,j, = y^N~~~ and 
^^3b3b3b = where Nj^j^j^ and Nj,j,j, (i.e., 

without the superscript M) are the total number of events 
dominated by the background processes. 
Ea. H12() can be simplified to: 



SR. 



M _ VN~~+W^, 
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therefore, since N, 



M 

jbjbjb 



M 



M 



cx Y^, it follows that SR. 
^/Y^. In particular, if Y\j cx tan/?, then SR\ 
l/tan2/3. 

The condition, 5R^^ > SR^l^ signals that new physics 
in the ratio between the ft.cc and hbb Yukawa couplings 
can be seen, with a statistical significance of 



NsD 



SR 



M 



SR^' 



(14) 



III. NUMERICAL SETUP AND BACKGROUND 



Tevatron: prU) > 15 GeV , |77(j)| < 2, 
LHC: ^ pT{j) > 30 GeV , |77(j)| < 2.5, 
where j is any of the three jets in the final 
state. 

2. The signal cross-sections always have one pair 
of jets (coming from the Higgs decay) that 
should reconstruct the Higgs mass. Thus, in 
order to improve the signal to background ra- 
tio, we require for both the signal and back- 
ground cross-sections that only one jet pair 
(out of the three possible jet pairs in the fi- 
nal state), will have an invariant mass within 
ruh ± 0.05to^, i.e., within ±5% of around 
ruh- That is, we reject events if the invariant 
masses of more than one jijk pair are within 
ruh — 0.05m/i < Mj.j^ < mu + 0.05m;i. This 
acceptance cut is found to significantly im- 
prove the signal to background ratio. 

3. We impose a common lower cut on the in- 
variant masses of all three jet pairs: Mj^j^ > 
ruh/'i'- This additional cut further improves 
the signal to background ratio. 

4. We require the final state partons to be sepa- 
rated by a cone angle of Ai? > 0.7, where here 



Ai? stands for -^Z /S.Tf 



A(p2 



• We use the MS running Yukawa couplings 

YbifJ'R) oc {mb{p.R)/v), Yc{i ^R) PC {rhcinRj/v), 
where rhb{fJ-F/.) and rhc{p.R) are MS running masses. 
We take a renormalization scale oi fiR — mh as our 
central value. In particular, fhb{mh) and rhc{mh) 
are calculated via the next-to-leading-order heavy 
quark MS running mass equation 0, using fhb{fhb) 
and fhc{rhc) as the initial conditions. This brings 
up some uncertainty corresponding to the allowed 
range of fhb{fhb) and fhc{fhc): 4 GeV < fhbifhb) < 
4.5 GeV and 1 GeV < fhc{fhc) < 1.4 GeV, see 
[Tol |. In what follows we use fhb{rhb) — 4.26 GeV 
and fhc{ific) = 1.26 GeV as our central values [lol |. 
which give the fhb{mh) and ific{mh) masses listed 
in Table n 



Before presenting our results let us describe the nu- 
merical setup: 

• All signal and background cross-sections are calcu- 
lated at leading order (tree- level) , using the COM- 
PHEP package with the CTEQ5L ^ parton dis- 
tribution functions. 

• Throughout our entire analysis to follow, the fol- 
lowing set of cuts are employed in both signal and 
background cross-sections: 

1. For both the Tevatron and the LHC, in order 
for the jets to be within the tagging region of 
the silicon vertex detector, we require all three 
jets to have the following minimum transverse 
momentum (px) and rapidity (jj) coverages: 



TABLE I: Running charm and bottom quark MS (NLO) 
masses at hr = mh for rUh = 100, 120 and 140 GeV, with 
the initial conditions fhb{fhb) — 4.26 GeV and ffic(fhc) — 1.26 
GeV. 



running 


charm and bottom MS 


masses 


fht{fnt) = 


4.26 GeV, me(me) = 1.26 GeV 


4[GeV]^ 


flR = 100 


Mii = 120 


fiR = 140 




2.94 


2.89 


2.86 




0.61 


0.6 


0.59 



At some instances we will refer to the "maximal 
signal" possible. This means that we are evaluat- 
ing the signal cross-sections with the initial con- 
ditions rfibifhb) = 4.5 GeV and fhc{mc) = 1.4 
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GeV, i.e., at their upper allowed values, which at 
liR = 100 GeV gives: mb(100 GeV) = 3.1 GeV and 
mc(100 GeV) = 0.68 GeV. 

• Unless stated otherwise, the parton distribution 
functions are evaluated with a factorization scale 
of hf = "TTih- Thus, we use a common factorization 
and renormalization scale /i = = and take 
/i = mh as our nominal value. The uncertainty 
of our signal cross-sections, obtained by varying 
the factorization scale ^f about its central value 
(fiF = iTT-h) within the range to/i/4 < fiF < '^irih, 
will also be investigated. 

• We do not consider the next-to-leading-order 
(NLO) contributions to a{qg qh). As was shown 
in 0, the NLO contribution to a{bg hh) can 
amount to an effective if- factor of about 1.5 at 
the Tevatron and about 1.2 at the LHC. Assum- 
ing that a similar ii'- factor applies also to a{cg 
ch), our signal R^^ is essentially insensitive to the 
NLO corrections since it involves ratios of these 
cross-sections. Moreover, SR^l^ is proportional to 

y/W/K'^, where and are the if-factors 
for the background and the signal three jets events, 
respectively. Thus, at NLO, the statistical signif- 
icance of our signal {Nsd = ^R^^ / ^Rti^) should 
be about a factor of j^I larger or smaller 
than our tree-level estimate, depending on the size 
of and K^. For ~ > 1, our tree-level 
prediction for Nsd is on the conservative side. 

• Since we require our signal to be composed out of 
events with exactly three jets in the final state (i.e., 
3 heavy jets, jhjhjh), we do not consider processes 
with 4 heavy jets or 4 6 jets in the final state, e.g., 
gg, qq ^ bbh — > bbbb. These type of sub-processes 
constitute a partial source of the K factor for the 

3 heavy jets signal cross-sections Moreover, 
as was further demonstrated in |3| for the case of 
Higgs-bottom associated production at hadron col- 
liders, even if we had relaxed our demand for ex- 
actly 3 jets in the final state to allow for 3 or 4 jets 
with 3 or 4 jet tags, the contribution of events with 

4 jets to would still constitute no more than 
10% of those with exactly 3 jets in the final state. 

Let us now discuss the background cross-sections. The 
irreducible background for the various three heavy jet fi- 
nal states, jhjhjh with /i = c or 6, is dominated by the 
QCD sub-processes bg — > bg — > bbb, bcc and eg ^ eg 
cec, ebb. Other processes like Z production followed by 
its decay, bg ^ bZ ^ bbb, bcc and eg ^ cZ ^ cec, ebb, 
and QED also contribute to the background. In what 
follows, this irreducible background is calculated using 
the COMPHEP package 7] , where all possible tree-level 
diagrams (QCD, Electroweak and QED) for the jhjhjh 
final state are included. In addition, due to the non-ideal 



efficiencies, e.g., the non-zero probability of misidentify- 
ing a light jet for a heavy jet, there is a reducible back- 
ground for the three heavy jet final states coming from 
sub-processes in which one, two or all three jets are light. 
Since one expects that ef , <C e^, e^, e^, ejj (see e.g., 
^3), the reducible background cross-sections, when mul- 
tiplied by ej* or are dominated by processes in which 
only one jet out of the three is light. Moreover, we find 
that, at both the Tevatron and the LHC, the one-light-jet 
reducible background is by far controlled by the gg —^ gbb 
and gg — > gee sub-processes, which will be therefore in- 
cluded in our background estimation. For example, the 
contribution of gg — *■ gbb to the jhjhjh final state is: 
^fhjhjh = ^aa-^gbb x <^'i{<^bY (the background signals will 
always be denoted by the superscript B). 

In Tables |n] and IIIII we list the background cross- 
sections at the Tevatron and at the LHC, respectively, 
where all cross-sections are calculated with the set of cuts 
1-4, described earlier. The background cross-sections are 
calculated with a factorization scale set to jiF — V^, 
where s is the square of the cm. energy of the hard 
cross-sections. 

TABLE II: Background cross-sections in [fb], for the Teva- 
tron with a cm. of ^/s = 2 TeV. All cross-sections are 
calculated with the following kinematical cuts (some defined 
as a function of mu): (1) PT{j) > 15 GeV, (2) |?7j| < 2, 
where j is any (heavy and light) of the three jets in the 
final state, (3) the acceptance cut that the invariant mass 
of only one jet pair Mj-j^ , i ^ k, out of the three possi- 
ble pairs of jets in the final state is within the mass range 
(mh-0.05mh) < Mj.j,^ < {mh + 0.05mh), (4) Mj-j^ > mh/2, 
for all three jet pairs, and (5) any two partons in the final 
state are separated by a cone of A_R > 0.7. Numbers are 
rounded to the 1% accuracy as was obtained from the COM- 
PHEP numerical sessions. 



Background cross-sections at the Tevatron 


[fb]4 / [GeV]^ 


ruh used for the kinematical cuts 


nih = 100 


m,i = 120 


nih = 140 


{gb bbb or bcc) 
{gc ebb or cec) 
^■■^(SS gbb or gcc) 


4.4 • 10^ 
7.2 • 10^ 
230 • 10^ 


1.8 • 10^ 
3.0 • 10^ 
94 ■ 10^ 


9.5 • 10" 
16.3 • 10^ 
41 • 10^ 



TABLE III: Background cross-sections in [fb], for the LHC 
with a cm. of ^/s — 14 TeV. All cross-sections are calcu- 
lated with prij) > 30 GeV, jr/jj < 2.5 and the additional 
kinematical cuts 3-5 as in Table dl (see caption to Table |hJ. 



Background cross-sections at the LHC 


[fb]j; / [GoV]^ 


rUh used for the kinematical cuts 


mh = 100 


mh = 120 


mh = 140 


a" {gb bbb or bcc) 
(gc ebb or cec) 
'^'^{99 gbb or gcc) 


133 ■ 10^ 
190 • 10^ 
5680 • 10^ 


111 • 10^ 
159 ■ 10=^ 
4720 ■ 10^ 


84 • 10^ 
120 • 10^ 
3540 • 10^ 
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IV. EXPECTATIONS FROM TWO HIGGS 
DOUBLET MODELS 

The signal cross-sections depend on the Higgs cou- 
plings and therefore on the underlying Higgs model. In 
what follows we will investigate the sensitivity of our sig- 
nal R'^' to various versions of a 2HDM. 

The most general 2HDM Yukawa term is given by 



hdd 


vSM 


X 


tan/3 , 


huu 


^ u 


X 


tan/? ; u = u or c , 


htt 


ySM 


X 


cot/3 , 


Add 


vSM 


X 


tan ^75 , 


Auu 


^ u 


X 


tan /375 ; u = u or c 


Att 


ySM 


X 


cot/375 . 



(17) 



(15) 



where Ql is the SU(2) left-handed quark doublet, 
uu and dji are the right-handed up and down quark 
SU(2) singlets, respectively, and $i_2 = icr2$| 2- Also, 
U^,U^, D^,D^ are general Yukawa 3x3 matrices in fla- 
vor space. The different types of 2HDM's are then cate- 
gorized according to the different choices of the Yukawa 
matrices U\ U'^ , , D'^ . 



A. The 2HDM for the top quark - T2HDM 

As an example of a Higgs sector that can give rise 
to an observable effect via , we consider first the so 
called "2HDM for the top " , in which the top quark re- 
ceives a special status [l^l • We will denote this model by 
T2HDM. In the T2HDM one defines 



Gij X {5ji + 5j2) 
Gij X Sjs , 
, 



(16) 



where G is an unknown Yukawa 3x3 matrix in quark 
flavor space. The large mass of the top is, thus, naturally 
accommodated in the T2HDM by coupling the second 
Higgs doublet ($2), which has a much larger Vacuum 
Expectation Value (VEV), only to the top quark. This 
model has therefore large tan/3 = V2/V1 by construction. 
That is, V2 is the VEV responsible for the top quark mass, 
while vi generates the masses of all the other quarks. 

Like any other 2HDM, the Higgs spectrum of the 
T2HDM is composed out of three neutral Higgs; two CP- 
even scalars and one pseudoscalar, and a charged Higgs. 
Choosing the basis a = f3, where a is the usual mixing 
angle between the two CP-even Higgs particles (see e.g., 
^3]), the neutral Higgs spectrum of the T2HDM is ro- 
tated such that it contains a SM-like neutral Higgs, Hsm, 
(with the SM couplings to fermions), a CP-even Higgs, 
h, and a pseudoscalar (CP-odd Higgs), A. The Yukawa 
couplings of h and A to down and up-type quarks in the 
T2HDM are: 



where d — d , s 01 b and Y^^'^ — mj /v, v — \/ v\~V v\ — 
246 GeV. 

We therefore see that while the Yukawa couplings of 
h and A to down quarks in the T2HDM have the same 
pattern as in the 2HDMII and the MSSM, i.e.. Yd oc 
Y^^^ X tan /?, their Yukawa couplings to a u and a c quark 
exhibit a different behavior, due to the special structure 
of the T2HDM's Yukawa matrices in (|16|l . In particular, 
Yc and Y^ are enhanced in the T2HDM by a factor of 
tan^ /? as compared to their values in the 2HDMII or in 
the MSSM, in which Y^ oc Y^^ x cot /3 for m = u, c or 
t. Notice also that Yi (in the T2HDM) remains the same 
as in the 2HDMII and the MSSM. 

Therefore, as will be shown below, since 8R^^ oc 
{YcjY}^'^ [see (lll(l ]. we can expect to have the follow- 
ing relations: bR^"^^^^ ~ 8R^^'^ cx {mc/mbY and 
gj^2HDMii^ gj^MSSM ^ SR™""^', when tan/3 is large. 
Note that in spite of the fact that SR^^'^ - SR™'^^ , 
due to the small production rate of a SM (or a SM-like) 
Higgs via the gluon-6 and gluon-c fusion channels, 5R^^^ 
is overwhelmed by the very large background (i.e., 5Rf^^) 
and is thus unobservable. 

In Tables IIVI and we give the signal cross-sections 
at the Tevatron and at the LHC, respectively, for the 
T2HDM with tan/? — 30 and evaluated at /x = mt as 
the central value. To appreciate the uncertainties cor- 
responding to variations of the factorization scale (/if), 
we also give in Tables HVI and El the signal cross-sections 
evaluated with /if = /4 (upper uncertainty) and with 
/iF = 2m/i (lower uncertainty), while keeping a fixed 
renormalization scale /i^ = nih- The effect of /if on 
the statistical significance of the signal will be discussed 
below. 

All cross-sections in Tables IIVI and [V] are calcu- 
lated with the set of cuts, as described in the previous 
section. [•^1 To a good approximation, the signal cross- 
sections given in Tables HVl and Ivl are essentially insen- 
sitive to whether the Higgs produced is h or the pseu- 
doscalar A. The difference between the two is the 1 ver- 
sus the 75 couplings to fermions, which amounts to sign 
differences in the sub- leading terms involving m?, or m,.. 

Using eqs. (0-® and we plot in Figs. Q] 

and|21the statistical significance {Nsd) of (Ji^'^^HDM (-^j^g 



[3] The relevant Lagrangian pieces of the T2HDM were implemented 
into COMPHEP. Also, the Higgs width was explicitly calculated 
within the T2HDM via COMPHEP. 
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LHC, s"'^=14 XeV Xevatron, s'"=2 XeV, L=30 [fb] ' 




I . 1 . 1 . 1 . 1 Q Wzizzl . 1 . 1 . 1 . 1 

lo 20 30 40 50 10 20 30 40 50 



FIG. 1: The statistical significance of the signal in the T2HDM {Nsd = SR™'^^^ /SR™'^^^) as a function of tan/3 and for 
ruh = 100, 120 and 140 GeV, at the LHC with integrated luminosity of 30 fb"^ and 100 fb"^ (left plot) and at the Tevatron 
with integrated luminosity of 30 fb"'^ (right plot). The upper solid curve on the right plot is for the case of maximal charm 
and bottom MS quark masses at fiR = m;, — 100 GeV (see discussion in section ITTT^ . All cross-sections were calculated for 
H = Hf = fJ-R = rrih and with the set of cuts 1-4 as outlined in section ITTTl 



TABLE IV: Signal cross-sections in [fb], for the T2HDM with 
tan/3 = 30, at the Tevatron with a cm. of — 2 TeV. 
All cuts are the same as for the background cross-sections 
at the Tevatron (see caption to Table HB . The upper and 
lower uncertainties correspond to fip ~ ^1^/4 and /if = 2mh, 
respectively. 



Signal cross- 


sections for the T2HDM with tan /? = 30 






Tevatron, ^/s — 


2 TeV 


[fb]J| / [GeV]^ 


m,h = 100 


ruh = 120 


mh = 140 




^ bbb) 




ooci+16% 






bcc) 


42+9% 


J^D-0_10% 


7+20% 
'-12% 




ebb) 


79+35% 
'^-14% 


9« /I +39% 


1 r, 1 +42% 
J-^-J-_i5% 




ccc) 


1+35% 
■^•-^-14% 


1 o'3+39% 





TABLE V: Signal cross-sections in [fb], for the T2HDM with 
tan/3 = 30, at the LHC with a cm. of ^ = 14 TeV. All 
cuts are the same as for the background cross-sections at the 
LHC. See also captions to Table IITTI and HVI 



Signal cross- 


sections for the T2HDM with tan /3 = 30 






LHC, y/s^U TeV 


[fb]j; / [GeV]^ 


mh = 100 


rrih = 120 


rUh = 140 


a^'^^^^^(36 


^ bbb) 


46000; 1%^° 


32400;^^:° 


21900:1% 




bcc) 


2000; 


1400;^f 


y^u_i% 


^T2HDM(^^ 


ebb) 


3050+^^° 


2120+3% 


1400+5% 


^T2HDM(^^ 


ccc) 


101+0% 


Qn + 5% 
^^-3% 


60^^^: 



signal within the T2HDM) , as a function of tan (3 for the 
LHC vifith an integrated luminosity of 30 fb^^ and 100 
fb^^ and for the Tevatron with an integrated luminosity 
of 30 fb^^. The statistical significance was calculated 
for tan/3 = 10, 20, 30, 40, 50 and interpolation was 
employed to obtain the results for other values of tan (3. 
As our nominal values, the following efficiency factors 



were chosen |Tl|: = 0.005, 0.01, = 0.1, el = 

0.6, = 0.8, with = 0.6 in Fig. □ and e'^' = 0.3 in 
Fig. El 

Evidently, 6R'^^^^^^ can be easily observed at the 
LHC for nifi — 100 — 140 GeV, with a significance of 
more than 3-sigma for tan /3 ~ C(20) if ejf = 0.6 and for 
tan/3 ~ 0(35) if — 0.3. For example, already at the 
low luminosity stage of the LHC, i.e., with L = 30 fb~^, 
a 5-sigma detection of 6R^^^^^^ will be obtained for 
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nih ~ 100 GeV if = 0.6 and tan/? ~ 20 or if = 0.3 
and tan/3 ~ 35. On the other hand, a 3-sigma detec- 
tion of 5BF^^^^^ will be possible at the Tevatron only 
if ^ 0.5 and if mh is around 100 GeV and tan/? >, 50. 
The maximal signal case [obtained when fhb{mh) and 
TOc(to?i) are evaluated with maximal fh{,{fhb) and ific(rfic) 
as initial conditions, see the discussion in section Ullj . 
is expected to give a 3-sigma effect at the Tevatron for 
ruh = 100 GeV and tan/3 > 45 and if ej! - 0.6. 

In Figs. 13 and 0] we give scatter plots which show the 
region in the — ej! plane that will allow a 3-sigma 
detection of 5R™°^^ at the LHC when tan/3 = 20, 
ruh = 100 - 140 GeV, ef = 0.005, = 0.01, = 0.1, 
and with a jet tagging efhciency of e\ = 0.6 in Fig. |3| 
or of — 0.4 in Fig. 0] Clearly, a 3-sigma detection 
of 6R^^^^^^ over the entire rrih mass range shown, will 
be possible at the LHC for a b-tagging efhciency of 60% 
(ffc = 0.6) [U and with e^, e^! as low as - 0.3 - 0.4 
if L = 100 fb-\ or with e^, e'^ ~ 0.5 - 0.6 if L = 30 
fb^^. Fig. ^ shows that for a lower b tagging efficiency 
of el = 0.4, the requirements of e^, ej- > 0.4 for L = 100 
fb-i and e^, ejf > 0.6 for L = 30 fb^^, will also sufhce 
for a 3-sigma detection. 

Finally, in Table IVTl we show the variation of the statis- 
tical significance of the signal {Nsd of 5R^^^^^^) for dif- 
ferent choices of the factorization scale, fip — ™/i/4, rrih 
and 2m/j. The numbers in Table IVll correspond to the 
LHC with an integrated luminosity of 100 fb~^ and are 
given for tan/3 = 30. Evidently, only mild changes 
(smaller than 5%) in Nsd are obtained when varying the 
factorization scale within the range rrih/A < fip 'i-'nih- 



TABLE VI: Comparison of the statistical significance in the 
T2HDM {Nsd = 5R™'^^' /SRl^^'^"') for three values of 
the factorization scale /if = mh/'^, mn and 2mh- Nsd is 
given for nih = 100, 120 and 140 GeV and tan/3 = 30, at the 
LHC with an integrated luminosity of ICQ fb~^. 



Nsd in the T2HDM with tan/3 = 30 


[GeV]^ 


LHC with C = 100 fb-" 


nih = 100 


mh = 120 


mh = 140 


M-F = 'm.h/4 


6.37 


5.15 


4.03 


/iF = rrih 


6.71 


5.14 


3.92 


fiF = 2mh 


6.51 


5.03 


3.77 



B. The 2HDMII and the MSSM 

Let us now examine the expected signals ^J^^dmii 
and SR'^^^^^ in the 2HDMH and in the MSSM, respec- 
tively. Here too, we assume from the outset that tan/3 
is large enough so that the non-standard Higgs can be 
observed in association with bottom quark jets at the 
hadron colliders under investigation (recall that if the 
neutral Higgs does not have the necessary tan /3 enhance- 
ment in its Yukawa coupling to the bottom quark, then 
our signal ceases to be effective). Thus, in what follows 
we will focus only on the large tan/3 case. 

The 2HDMII Yukawa couplings follow from the 
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LHC, s" =14 TeV, tan|3=20, e, =0.6, e;=0.1, e,"=0.005, e,"=0.01 



L=100 [fb]" 



L=30[fb]" 




FIG. 3: Scatter plot for the LHC (Vi = 14 TeV) in the - plane, for tan/3 = 20, e\ = 0.005, e'l = 0.01, = 0.1 and 
= 0.6. The shaded area allows an above 3-sigma detection of /j^^hda/ ^j^^ LHC with integrated luminosity of 30 fb"'^ 
(right column) and 100 fb~^ (left column) and for — 100, 120 and 140 GeV on the first, second and third rows, respectively. 
The shaded area enclosed to the right of the solid lines corresponds to the conditions > ej? and > e|j. All cross-sections 
were calculated for fi — fip = /J^r = nih and with the set of cuts 1-4 as outlined in section ITTTl 



Yukawa potential in 1)15(1 by setting — and — 0. 
Thus, in this model <i>2 is responsible for the mass gen- 
eration of the up-type quarks while the down-type quark 
masses are generated through the VEV of $i. Denot- 
ing again the three neutral Higgs species of the 2HDMII 
by h, H (the two CP-even scalars) and A (the CP-odd 
Higgs), their couplings to fermions are given by (see e.g., 

MY- 



hdd 


vSM 


X 


[sin(/3 - 


a) 


— tan P cos(/3 — 


a)] 


huu 


^ u 


X 


[sin(/3 - 


a) 


+ cot /3 cos(/3 — 


a)] 


Hdd 


vSM 


X 


[cos(/? — 


a) 


+ tan/?sin(/3 — 


a)] 


Huu 


ySM 

^ u 


X 


[cos(/3 — 


a) 


— cot /? sin(/3 — 


a)] 


Add 


vSM 
^d 


X 


tan/375 
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LHC, s" =14 TeV, tan|3=20, e, =0.4, e;=0.1, e,"=0.005, e,"=0.01 



L=100 [fb]" 



L=30[fb]" 




FIG. 4: Same as Fig. El for = 0.4. 



Auu : Y°^'' X cot/375 , (18) 

where diu) stands for a down(up) quark and a is the 
mixing angle in the CP-even Higgs sector. We will study 
three representative cases: 

Case 1: cos(/3 — a) ^ 1 or a ^ /3. 

Case 2: cos(/? — a) ^ or a ^ /3 — 7r/2. 

Case 3: cos(/3 — a) ~ sin(/3 — a) ^1/ \/2 or a ^ /3— 7r/4. 

The charm and bottom Yukawa couplings and their 
ratio, Yc/Y},, are given in Table IVlTI for the three cases 



above and for each of the neutral Higgs bosons /i, H and 
A. 

The expected signals and their statistical significance 
for the three limiting cases and for the three neu- 
tral Higgs particles of the 2HDMH are compared to 
the T2HDM case in Table IVIIll Clearly, even in 
the more favorable cases in which Nsd{'2HDMII) ^ 
Ns d{T2H D M) / tan^ f3, the expected statistical signifi- 
cance in the 2HDMH case is still at least two orders of 
magnitudes smaller than the one expected in the T2HDM 
when tan 10. Therefore, based on the results ob- 
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TABLE VII: The charm and bottom Yukawa couplings 
and Yb scaled by their SM Yukawa coupling Yf" and V/*', 
respectively, and their ratio IFc/yf)!, in the 2HDM of type 
II. The Yukawa couplings of h, H and A are given for the 
three cases: (1) cos(/3 — a) — > 1, (2) cos(/3 — a) ^ Q and (3) 
cos(/3 — a) ~ sin(/3 — a) ~ l/\/2, see also text. 



Yc and n in the 2HDMII 


Case 


Higgs 
particle 






\Yc/Yb\ 




h 
H 


1 


tan P 
1 


rric 


1 


tan (3 

1 


m(j tan-^ f3 
m,c 
i^h 




A 


1 


tan /3 


rric 




tan /3 


rrtf, tan-^ 




h 


1 


1 


m,c 


2 


H 

A 


1 


tan P 
tan /3 


rric 


tan [3 
1 


rrLfj tan-^ 




tan /3 


rrif, tan-^ 




h 
H 
A 


1 


tan 13 


m,c 


3 


V2 
1 


~7T 

tan /3 


rrify tan 
rric 




tan P 


mi, tan 
rric 




tan /3 


rrif, tan-^ 



tained for the T2HDM scenario, we conclude that, no 
signal of R is expected to be observed at the LHC or at 
the Tevatron if the Yukawa Higgs sector is controlled by 
the 2HDMII. Hence, a measured signal of SR at these 
hadron colliders will rule out the 2HDMII Higgs scenario 
with many standard deviations. 



Case 1: ;S m^"=^, where m^J*"^ ~ 120 or 135 GeV, 
is the maximal allowed mass of the lighter CP-even 
Higgs after radiative corrections are included in the 
CP-even sector, depending whether one takes the 
minimal or the maximal stop mixing scenario, re- 
spectively j|14ij. In this case cos(/3 — a) ^ 1. 

Case 2: m\ » m|, the so called decoupHng limit. In 
this limit cos(/? — a) — > 0. 

These two MSSM limiting cases for large tan (3 remain 
valid also after the radiative corrections to the Higgs sec- 
tor are included thereby shifting the value of the 
mixing angle a (the higher order contribution to a can, 
on the other hand, cause a large shift to the lighter CP- 
even Higgs mass, nih)-^'^^ It should also be noted that, 
in the large tan/3 case, cos^(/3 — a) approaches zero very 
rapidly as niA is increased. In particular, an order of 
magnitude drop of cos^(/3 — a) (from 1 to 0.1) is spanned 
over no more than about 10 GeV mass range of rriA |l5j| . 

Since cases 1 and 2 in the MSSM Higgs sector are 
equivalent to cases 1 and 2 of the 2HDMII framework, 
they have the same Yukawa couplings pattern as given 
in Table IVlTl Thus, applying the results obtained in the 
2HDMII to the MSSM case, we conclude here too that 
gj^MSSM observed either at the Tevatron or 

at the LHC. Reversing the argument, a measured signal 
of 5R at these hadron colliders will rule out the MSSM. 



TABLE VIII: The expected 2HDMII signals, 5Bf""^^^' , and 
their statistical significance, Ng^^^^''^ , scaled by the corre- 
sponding values in the T2HDM, for cases (1) cos(/3 — a) —> 1, 
(2) cos(/3 - a) ^ and (3) cos(/3 - a) ~ sin(/3 - q) ~ 1/^2, 
and for the three neutral Higgs of the model h, H and A. 



Expected signals in the 2HDMII 


Case 


Higgs 
particle 






j.r2HDMII 
"SD 








1 


h 
H 
A 


i 


1 

tan^ P 
1 


1 


tan* l3 
1 
1 


tan* /3 

tan^ 
1 


tan* P 


tan* 


2 


h 
H 

A 


1 

1 


tan-' P 
1 
1 


i 

tan^ p 
1 


tan* l3 


tan* p 
1 


tan* |3 


tan* 


3 


h 
H 

A 


1 


1 
1 
1 


i 


tan^ (3 


tan^ 
1 


tan^ (3 


tan^ 
1 


tan* a 


tan* 



The 2HDMII Higgs framework also underlies the 
MSSM Higgs sector. However, due to the supersym- 
metric structure of the theory, the MSSM's Higgs sector 
is completely determined at tree-level by only two free 
parameters, conventionally chosen to be tua and tan/3. 
That is, the mixing angle a is fixed by and tan/3 at 
tree level. Concentrating again on the large tan /3 case, 
in the MSSM one can distinguish two limiting cases: 



V. SUMMARY 

To summarize, we have proposed a signal, 5R [see (|ll|l ] , 
based on counting the number of three heavy (c or b) jets 
events versus the number of events with three b jets, in 
processes in which the neutral Higgs is produced in as- 
sociation with a single high pT c or 6 jet. This signal 
assumes that the Higgs Yukawa coupling to the b quark 
is enhanced, say by a large tan/3 in multi Higgs doublet 
models, and that the neutral Higgs will therefore be ob- 
served in association with b jets at future hadron collider 
experiments. 

This signal was calculated in the framework of multi 
Higgs models which have at least one neutral Higgs with 
an enhanced Yukawa coupling to the b quark when tan /3 
is large. We have found that in such cases, the signal to 
background cross-section ratio is typically (after applying 
some useful kincmatical cuts) S/B ^ 0.1 — 1 at the LHC 
and S/B ^ 0.05 — 0.3 at the Tevatron, depending on the 
value of tan/3. 

The measurement of such a signal will require an efh- 
ciency for distinguishing a c jet from a light jet at the level 



[4] There are also vertex corrections to the Higgs-fermion Yukawa 
couplings which do not depend on the mixing angle a. However, 
these have a negligible effect in t he up quark sector, and even 
more so for large tan f} values Il6l . 
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of about e'l - 20% - 30% or - 50% - 60% at the LHC 
or at the Tevatron, respectively. Such efRciencies seem to 
be somewhat higher than what has been attained in the 
simulations to date (e^? ~ 10%) ^3- We have shown that 
if such c jet tagging efficiencies are acomplished, then our 
signal will be very efficient for probing the ratio between 
the charm and bottom Yukawa couplings, Yc/^b, thus al- 
lowing a deeper insight into the Higgs Yukawa potential. 
This, in turn, will be useful for categorizing the theory 
that underlies the Higgs sector. It should be noted, that 
our predictions for the Tevatron rely on an accumulated 
yearly luminosity of 30 inverse [fb], which is, at present, 
considerably higher than the expected Fermilab's run II 
luminosity. 

It should also be noted that the background estimate 
made for our signal was based on a low probability for 
misidentifying a light jet as a heavy jet, i.e., e\' - 0(1%). 
While that value is comparable to the present estimate 
for e\ (or even a little conservative), it is somewhat op- 
timistic for which is not yet well studied by present 
detector simulations. A more realistic value for the dis- 
tinction between a light jet and a charm jet, based on the 
technology of today, would be ~ 0(10%). Nontheless, 
our background estimate (and therefore also our estimate 
for the significance of the signal) holds roughly for values 
of el not larger than about 10%. For at the level of 
tens of percents there may well be other processes, e.g., 
gg — > ggg, that may alter our signal to background es- 
timate to the level that the observability of our signal 
becomes difficult. Another potential problem to the de- 
tection of our signal may arise if the trigger algorithm 



for b-jet tagging requires a minimum transverse momen- 
tum cut at the level of 100 GeV and above and the pt 
cut of 30 GeV we used for the LHC and 15 GeV for the 
Tevatron proves to be too optimistic. In that case also, 
the signal proposed in this paper may be degraded and 
difficult to observe due to the large QCD background. 

If the difficulties mentioned above can be surmounted 
and such a signal is detected at the Tevatron or at the 
LHC, then the popular MSSM and the 2HDMII can be 
ruled out with many standard deviations, since in these 
theories the ratio Yc/Y\j is too small for our signal to 
have any measurable consequences. On the other hand, 
we have shown that within a version of a 2HDM - the 
"2HDM for the top" (T2HDM) - in which the large mass 
of the top quark is naturally accommodated for large 
tan /?, our signal can be easily observed at the LHC within 
the entire relevant mass range of the neutral Higgs if 
tan/3 > 20 and ~ 20% - 30%. In addition, if the neu- 
tral Higgs of the T2HDM has a mass around 100 GeV and 
tan P ^ 50, then our signal may also give an effect with 
more than 3-sigma significance at the Tevatron provided 
that ~ 50% - 60%. 
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